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t OMPARABIIJTY STUDY OF ARMhD SliRVICbS VOCATIONAL 
APTITUDE BATTBRY SCORKS FROM ANSWER SHEET AND 
ANSWER C ARD ADMINISTRATION 



I INTRODUCTION 

The Armed Services Vocational Aptitude 
Battery (aSVAB), described by Vitola and Alley 
( 1^)68) and BayrotT and Fuchs( 1^70). is currently 
operation;! for testing of high school seniors in 
voluntarily participating high schools throughout 
the United States. The current form of ASVAB 
requires the use of a DIGITEK answer sheet, 
scored on a DIGITEK DM100 optical scanner 
located at Randt)lph Air Force Base, Texas. 

With i>nly one form of the ASVAB available, 
test security requirements dictated that alternate 
forms of the test be ftroduced. Consequently, 
ASVAB-2 and -3 were developed and were normed 
using a geographically representative sample tested 
at specified Armed Forces Entrance and 
Examining Stations (AFEES). Beginning in 
September 1^73, ASVAB-2 will be used for high 
school testing, with ASVAB-3 reserved as the joint 
services qualifying entrance test. 

Fornv2 of the ASVAB was standardized on the 
DIGITEK answer sheets, but with the beginning of 
high school testing during the fall of 1973, the 
DIGITEK DM'lOO scoring capability at Randolph 
AFB wilt lie inadequate to handle the anticipated 
load. 

A revised answer form, printed on a three part 
IBM card, was designed to replace the answer 
sheet, will a resultant reduction in the amount of 
space allov/ed examinees for answer marking. The 
revised form was designed for scoring via the 
Hewlett-Packard model WAOO/D mark sense 
reader also at Randolph AFB. 

A preliiiinary study, to compare the two test 
response i;cording methods, was conducted at 
Lackland /.FB. Texas, to determine whether there 
was test ni rm slippage due to answer form change. 



II. PRO( tDl K! 

Two matched samples* each consisting of 248 
male I'SAF basic airmen assigned to Lackland 
AFB for processing and training, were admin- 
istered Fo.m 2 of the ASVAB. One sample was 
administcrt l Form 2XC (answer card format) 
while the rther was tested on ASVAB Form 2X 
(answer shtet format). The samples were matched 



on Armed Forces Qualification Test (AFQT) 
scores contained in their records. Matching was 
completed for as many percentile scores as 
available from within the total number of airmen 
tested. Distribution of AFQT deciles for the 
samples is given in Table 1 . 

Testing was accomplished under normal testing 
room conditions. ASVAB-2XC and •2X were 
administered alternately during morning and after- 
noon testing sessions; each form was administered 
an equal number of li^nes during each half-day to 
preclude time-of-dav effects. Scoring was accom- 
plished at Randolph AFB on the DIGITEK 
DM-100 for Form 2X and on the Hewlitt-Pack ud 
mark sense reader for Form 2XC. 

Two samples were processed separately. Means, 
standard deviations, and Pearson Product Moment 
intercorrelatk)n matrices were computed for 
twenty variables (these included years of educa- 
tion, AFQT, four cimiposites from the Airman 
Qualifying Examination, data on completion of 
specific high school courses, and all nine tests of 
the ASVAB). in addition, each of the two samples 
was divided into eight subsamples based on AFOT 
decile, and means and standard deviations of the 
ASVAB tests were computed for each resulting 
subsample. Raw score distributions on the nine 
ASVAB tests were prepared for each of the two 
samples. 

For each ASVAB test, a two-way classification 
analysis of variance was computed to assess effects 
of format and of the format x AFQT decile inter- 
action on scores. In this regard, it is noted that a 
significant effect for AFQT decile would be 
expected in all cases as a fun<^tion of correlation 
between AFQT and the test. Test norms, as 
previously established, are appropriate if in- 
consequential effects are found both for format 
atid for the interaction of AFQT decile with 
format; a significant F ratio on either of these 
effects, however, would suggest normative adjust- 
ments as a consequence of format change. 

ilL RESULTS 

Although the answer sheet sample (2X) 
contains fewer cases with education beyond high 
school than does the answer card sample (2XC), 
the average educational levels are comparable* 
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Answer sheei sample airiiion (2X) indicated 
CDnipletion for more liigh school math and science 
ci»urses than did answer card sample cases (2XC) 
even though sample 2XC had more airmen 
completing \2 or more years of education (Tables 
2 and 3). 

The two samples were matched on AFQT score; 
each case in sample 2X was matched with a sample 
2XC case witli the same AFQT sci»re. AFQT scores 
appear to be fairly evenly distributed across the 
deciles 20 tlirougli 90; there were no scores below 
20. Actual score range was 21 ihrougli 98. Enlist- 
ment is m>t authorized (xn mental category V 
(AFQT score 0 • 9) applicants, and recent policy 
has also excluded those scoring below 21 on 
AFQT. The percentage of airmen in the samples' 
AFQT categories I through IV appear to be quite 
Similar to the distribution tor recent first term 
I'SAF airman accessions. (Unpublished data. See 
Table 4 for AFQT category distributions.) 

Further evidence of comparability of the two 
samples was found in the distribution of AQE 
« percentile scores. While statistically significant 
differences were not found between the two 
groups on any of the four AQE aptitude 
composites, there were small mean differences in 
favor of the answer sheet sample; the largest of 
these were on the Administrative (2,27) and 
Elect ronics (1.77) Als. Distributions, means, and 
standard deviations of the AQE composites are 
shown in Table 5. 

Minor differences. of no significance were noted 
among the intercorrelations for the two samples. 
This serves as further evidence that format change 
did not materially afiect the battery (Table 6), 

Table 7 shows, for both samples, means and 
standard deviations for each ASVAB test. In 
addition, no significant ASVAB test mean 
differences were found; the largest difference (.62) 
is on space perception. Thus, it can be seen that 
the formal cliange makes no difference in average 
performance. 

Outcomes of the nine two-way classification 
analysis of variance problems are summarized in 
Table 8. As expected, higlily significant F ratios 
were obtained in all cases for AFQT decile, 
reflecting ct>rrelat ion between the test and AFQT. 
in no instance was there a statistically significant 
answer format main effect; this finding is 
cimsistent with tlie quite small overall mean 
diflerences between the two samples (largest 
difference was .62 on space perception). Signifi- 
cant format x AFQT decile effects were found for 
two of the nine tests. This interaction was 
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significant at the 01 level for word knowledge und 
at the .05 level for coding speed. Interestingly; the 
largest interaction effect would have been 
expected on coding speed since it is a closely 
timed test of sitnple clerical speed and accuracy; 
consequently, a change in si/e of the response 
form had greater potential tor allecling 
perfonnance of '1iigh intelligence" and "low 
intelligence*' subjects differentially. An inspection 
of the decile-by-decile group means for ilie two 
format groups (Table 9) reveals that in AFQT 
decile 20-29, mean coding speed performance of 
the ansv/er sheet group is 6.16 raw score points 
higher than that of the answer card group, and 
that in decile 50-59, a difference of about the 
same magnitude occurs, but in favor of the answer 
card group. Other decile group means for coding 
speed are approximately equal. In the case of the 
word knowledge test, format group means differ in 
favor of the card format group by a little more 
than three raw score points in ihe AFQT decile 
50-59 and by a little over two raw score points in 
favor of the answer sheet format in the AFQT 
decile 60-69. Other decile group means are approx- 
imately equal. 

These two significant interaction effects would 
suggest possible differential effect of test response 
format change on test performance as a function 
of level of AFQT performance. In both format 
samples, correlation between AFQT and coding 
speed is quite modest (low .30s), and correlation 
between AFQT and word knowledge in both 
samples is in the low .50s. AFQT has served as the 
standard against which ASVAB tiornis are 
calibrated, but with these relatively modest 
correlations, it is possible that part of ttie within 
AFQT decile difference washes out in conversions 
utilizing the full samples (as a function of AFQT 
decile distributional overlap). However, this 
cannot be cLarly detennined from these data 
because of the truncation of the samples on AFQT 
(approximately the bottom 20 percent of the 
normative reference population is not represented 
here). 

Raw score distribution for both groups on all 
nine ASVAB tests are presented in Tables lu and 
1 1 , It is noteworthy that the ASVAB test 
distributions of the two format samples differ very 
little from each other. 



IV. CX^NCLUSIONS AND Rl (OMMI NDATIONS 

The resuhs of this, study suggest compaiable 
ASVAB score results usitig either the answer card 



ERLC 



6 



or the IMCilll K aiiswci slieet. Presently estab- 
lished Ar>VAB f'orm 2 norms may be satisfactor) 
for answer card administration. Although a signifi 
cant fomiat by AI'QT decile interaction was found 
on coding speed and wiird knowledge, overall 
means, s indard deviations, and intercorrelations 
for the two fc»rniat ciMiditionsare nearly the same. 
This sugj^ests that, despite the significant inter- 
action oi; these two tests, conversion tables may 
be affected very little if at all. 



Tahle I. Sample Distribution 
by AFQT Decile 





Sample 


* 






AFQT Decile 


2XC 


2X 


Total 


Percent 




35 


35 


70 


14.11 




28 


28 


56 


1 1.29 


70-7') 


30 


30 


60 


12.10 


60-6y 


27 


27 


54 


10.89 


50-59 


33 


33 


66 


13.31 


40-49 


33 


33 


66 


13.31 


30-.W 


37 


37 


74 


14.91 


20-29 


25 


25 


50 


10.08 


Total 


248 


248 


496 


100.00 



*Thc two sa triples are evenly distributed across the 
range of AFQT scores. No store lower than 21 nor higher 
than 98 was reported. The two samples are pre- 
ctsley matched do AFQT scores. 



Table 2. Distribution of Educational 
Level 



Sample* 



Education 
(years) 


2KC 


2X 


Total 


8 


•> 


3 


5 


9 


4 


9 


13 


10 


23 


19 


42 


11 


28 


37 


65 


12 


147 


145 


292 


!3 


20 


21 


41 


14 


15 


10 


25 


15 


3 


3 


6 


16 


6 


1 


7 


Total 


248 


248 


496 


Mean 


11.96 


11.76 


1 1.86 


S.D. 


1.2? 


1.16 


1.21 



•A Mf „ ot* difference between the sample means 
revealed no agnifitant difference (t - 1.83). 



Because nf AFQT truncation in the data for 
this study, a final determination of whether 
conversions for the ASVAB tests are affected hy 
answer form format change must await completion 
of a full standardization study. Data for such a 
study have been collected and are being analyzed. 
It is recommended that present conversions for 
ASVAB Form 2 continue in use until completion 
of this standardization study. 



Table 3. Number of Cases Reporting 
Completion of High School Mathematics 
and Science Courses 



Sample* 



Course 


2XC 


2X 


Algebra 


154 


162 


Geometry 


95 


106 


Trigonometry 


29 


40 


Chemistry 


63 


76 


Physics 


34 


42 



*C:hi square tests revealed no significant differences 
between the two groups. 



Table 4, AFQT Mental Category 
Distribution of Samples 



Category 2XC 2X Total Pereant* 



1 



(93-100) 


23 


23 


46 


9.27 


li 










(65-92) 


80 


80 


160 


32.26 


HI 










(31-64) 


116 


116 


232 


46.77 


IV 










(10- JO) 


29 


29 


58 


11.69 


Total 


248 


248 


496 


99.99 



'Percentage uf Basic Airmen in each mental category 
is similar to the percentage in each category enlisted Into 
USAF during recent years (July 69 through September 
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Table 5. Distribution of AQE Scores 



sample 2XC Sample ax 



Percentile 


Ac) in 




Qen 


iVieon 


Adm 


Elect 


Gen 


Mech 


95 


9 


10 


IS 


11 


8 


14 


16 


1 1 


90 


•> 


to 


6 


13 


7 


18 


8 


Id 


K5 


It 


15 


6 


13 


12 


1 1 


7 


19 


80 


4 


20 


8 


s 


10 


'>'> 


16 

1 VI 


1 7 
1 J 


75 


K 


11 


16 


14 


6 


10 


A 

Vi 


S 

o 


70 


15 


13 


18 


8 


17 


7 


21 


13 


65 


17 


5 


IS 


10 


12 


10 


16 


13 


60 




29 


"»-> 


42 


26 


24 


23 


32 


55 


14 


31 


18 


3S 


12 


35 


15 


21 


50 


30 


18 


25 


25 


25 


18 


26 


22 


45 


21 


28 


33 


16 


20 


14 


35 


16 


40 


22 


17 


44 


7 


25 


25 


31 


15 


35 


16 


3 


11 


11 


21 


4 


13 


14 


30 


6 


IS 


5 


4 


15 


13 


5 


7 


25 


19 


10 


0 


14 


9 


12 


5 


6 


20 


20 


am 


4 


8 


13 


7 


4 


8 


15 


4 


2 


1 


11 


3 


1 


1 


10 


10 


8 


3 


1 


, 0 


6 


2 


0 


4 


05 


0 


4 


0 


1 


0 


0 


0 


1 


01 


0 


2 


0 


0 


1 


1 


0 


1 


Total 


248 


248 


248 


248 


248 


248 


248 


248 


Meun 


49.25 


56.44 


56.43 


55.89 


51.52 


58.21 


57.34 


56.51 


S.D. 


21.25 


21.68 


18.24 


21.14 


21.38 


21.80 


19.06 


22.71 
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Table 7. ASVAB Means and Standard Deviations 
for Samples 2XC and 2X 



Sample 



2XC 2x 



Test 


Mean 


so 


Mean 


so 


('i)iling Speed 


43.60 


11.42 


43.87 


11.95 


Word Knowledge 


14.95 


4.26 


14.91 


4.59 


Aridunetic Reasoning 


15.00 


5.36 


14.72 


5.43 


Tool Knowledge 


15.11 


5.30 


15.19 


5.05 


Spjce Perception 


16.68 


5.05 


17.30 


5.07 


Mechanical Comprehension 


14.98 


4.58 


15.40 


4.72 


Shop Information 


15.67 


4.70 


15.26 


4.57 


Automotive Information 


15.99 


t.68 


16.00 


4.8S 


Electronics Information 


15.06 


435 


15.49 


4.94 
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Table S. Analysis of Variance Summary 



source 


ss 


df 


MS 


F 


p 


ss 


df 


MS 


F 


p 






Coding Speed 








Word Knowledge 




Total 


70,401.43 


495 








8,712.67 


495 








Format 


18.97 


1 


18.97 


.16 


ns 


.20 


1 


.20 


.017 


ns 


Decile 


10,078,^3 


-7 


1 .439.75 


11.80 


<.001 


2.755.13 


7 


393.59 


33.05 


<.00l 


Format x 






















Decile 


L72I.84 


7 


245.98 


2.02 


<.05 


242.25 


7 


34.61 


2.90 


<.0I 






480 


122.05 






^ 7 1 S HQ 


480 


11.91 










Arithmetic Reasoning 






Automobile Information 




Tin;il 


I4,450.6X 


495 








1 1 .263.00 


495 






• 


Format 


y.88 


1 


9.88 


.60 


ns 


.06 


1 


.06 


.003 


ns 


Decile 


6.336.65 


7 


905.24 


55.06 


<.00l 


2.540.38 


7 


362.91 


20.20 


<.00I 


Format x 






















Decile 


215.12 


7 


30.73 


1.87 


ns 


98.14 
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Table 10. Raw Score Distributions on Coding Speed* 

(Samples SSC and 'JXj 













2X 


Soorc 


2XC 


2X 


So ore 


2XC 


2X 


Score 


2XC 2X 


80 






64 




"> 


48 


12 


9 


32 


7 


9 


16 


1 


79 






63 


3 


2 


47 


8 


9 


31 


4 


•) 


15 


1 


78 


1 




62 


1 




46 


6 




30 


1 


1 
1 


1 A 

14 


1 
1 


77 
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1 


4 
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4 
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3 


4 


13 


1 


7(1 


I 
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7 
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12 




75 
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5 
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3 
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8 
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1 
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21 




2 
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7 


36 


7 
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51 
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5 
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8 
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2 


50 
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34 


2 


7 


18 
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65 






49 


7 


8 


33 


4 


6 


17 


1 


2 


1 





Total N = 248 248 

*Siur ? nngc is from 0 to 100. Scoring formula: Right only 
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Tahlc I / Riiw Score* Distributions for ASVAB Tests 2 Tlirough 9 

(Sampler SXC and 2 \) 
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*Sci)ring formulae: R-W/3. 
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